Standard web and internet searches typically cover a small portion, perhaps only about ten percent, of the information that is accessible via the web. There are various reasons for this limitation. One such reason relates to scalability. For instance, significant costs, effort and infrastructure are associated with storing and indexing data, keeping information fresh and accessibly available. For information that is accessed relatively rarely, the costs may seem excessive in relation to the apparent demand, usefulness, etc. of the information. Doing this for all information that may be accessible via the web could be cost prohibitive.
Second, standard web search engines are deterred from accessing much of the information on the web because standard web crawlers, which help to build the indexes used by the search engines for searching, may lack the ability to get at that information. For example, a web crawler may not be able to access a database or a website that requires positive access enablement, such as password authentication or other security measures.
Finally, standard web search is designed for breadth and popularity, spanning the upper stratum of more popular information. While designed, in a sense, to return search results that may most probably please the most users, most of the time, many standard web search results may lack relevance for significant numbers of users. Seeking more relevant search results, vertical searching may be used, which is more specific than standard web search, being oriented more for specific domains and topics. However, users of vertical searching issue queries to and retrieve information from each search engine separately. With a relatively small selection of search engines, this practice may be more or less manageable. However, with many vertical search engines, it may soon become inefficient and expensive.
Based on the foregoing, it would be useful to perform vertical searches more effectively.
The approaches described in this section are approaches that could be pursued, but not necessarily approaches that have been previously conceived or pursued. Therefore, unless otherwise indicated, it should not be assumed that any of the approaches described in this section qualify as prior art merely by virtue of their inclusion in this section.